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Abstract 

Background: There is a need for more Comparative Effectiveness Research (CER) on Chinese medicine (CM) to 
inform clinical and policy decision-making. This document aims to provide consensus advice for the design of CER 
trials on CM for researchers. It broadly aims to ensure more adequate design and optimal use of resources in 
generating evidence for CM to inform stakeholder decision-making. 

Methods: The Effectiveness Guidance Document (EGD) development was based on multiple consensus procedures 
(survey, written Delphi rounds, interactive consensus workshop, international expert review). To balance aspects of 
internal and external validity, multiple stakeholders, including patients, clinicians, researchers and payers were 
involved in creating this document. 

Results: Recommendations were developed for "using available data" and "future clinical studies". The 
recommendations for future trials focus on randomized trials and cover the following areas: designing CER studies, 
treatments, expertise and setting, outcomes, study design and statistical analyses, economic evaluation, and 
publication. 

Conclusion: The present EGD provides the first systematic methodological guidance for future CER trials on CM 
and can be applied to single or multi-component treatments. While CONSORT statements provide guidelines for 
reporting studies, EGDs provide recommendations for the design of future studies and can contribute to a more 
strategic use of limited research resources, as well as greater consistency in trial design. 
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Background 

Chinese medicine (CM) includes a broad range of medical 
practices that have many of their roots in China and share 
common theoretical concepts. According to the descrip- 
tion by the National Center for Complementary and Al- 
ternative Medicine (NCCAM) CM 'encompasses many 
different practices, including acupuncture, moxibustion 
(burning an herb above the skin to apply heat to acupunc- 
ture points), Chinese herbal medicine, tuina (Chinese 
therapeutic massage), dietary therapy, and tai chi and 
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qigong (practices that combine specific movements or 
postures, coordinated breathing, and mental focus) [1]. 

In general, CM follows a theoretical framework, and the 
etiology and pathogenesis of CM uses its own termin- 
ology. The processes of diagnoses and interventions of this 
medical system are different from those in conventional 
medicine, and both are guided by traditional principles of 
CM. CM is often used as a multi-component treatment in 
which cultural, philosophical, historical, temporal, and 
geographic aspects as well practitioner training, all influ- 
ence its heterogeneity. 

From the practitioner's perspective, CM diagnoses (Tor 
example, bian zheng), often also called CM patterns or 
CM syndromes differentiation, inform CM interventions. 
To date, treatment individualization according to the 
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CM diagnoses seems to have very little clinically relevant 
impact on the outcome of acupuncture treatment in 
clinical studies [2-4], whereas it might be more relevant 
for clinical trials of CM pharmacotherapy [5,6], although 
evidence is still scarce. In practice, CM treatment is often 
individualized, and because CM diagnoses may change 
over time, interventions can also change during the course 
of treatment. Currently in China, standardization of CM 
diagnoses and treatment for practice and research is 
emphasized, whereas in the West, a trend toward more 
individualization in research protocols and in practice is 
observed. 

Aim of the document 

This document provides consensus advice for the design 
of comparative effectiveness research (CER) trials in CM 
for researchers. CER is the generation and synthesis of 
evidence that compares the benefits and harms of alter- 
native treatment options to prevent, diagnose, treat, and 
monitor a clinical condition or to improve the delivery 
of care. The purpose of CER is to assist consumers, cli- 
nicians, purchasers, and policy makers to make informed 
decisions that will improve health care at both the indi- 
vidual and population levels [7] . 

CER broadly aims to ensure more adequate design and 
optimal use of resources in generating evidence for CM 
to inform stakeholder decision-making. These consensus 
recommendations can be applied to single and multi- 
component interventions. They are based on the assump- 
tion that a reduction of internal validity can be justified 
in order to increase authenticity of the intervention and 
setting, thereby enhancing generalizability, relevance, 
feasibility and timeliness of research results. 

Methods 

The development of the Effectiveness Guidance Docu- 
ment (EGD) followed a structured and predefined consen- 
sus process, which included a pre-workshop online survey 
(April 2012), a consensus workshop (19 May 2012 in 
Portland, OR, USA), and three written Delphi rounds 
(August 2012, January 2013 and May 2013) utilizing writ- 
ten comments to finalize the document. 

Multiple stakeholders were involved in the consensus 
process for this EGD to balance aspects of internal and 
external validity in the recommendations. Participants of 
the workshop had the following backgrounds: one CM 
patient, one health insurance representative, nine experts 
in CM with experience in both CM practice and CM re- 
search (two from China, two with a Chinese background 
living in the USA, four from the USA, and one from 
UK), and 6 methodologists (with backgrounds in clinical 
research, statistics or epidemiology, 5 of them with experi- 
ence in CM research). The consensus meeting utilized 
presentations, large group discussions and an adapted 



world cafe methodology. The world cafe method, as devel- 
oped by Brown and Isaac, is a simple, effective, and flex- 
ible format for facilitating large group dialogue [8]. It has 
been used in the development of prior EGDs [9] to foster 
collaborative dialogue, knowledge sharing, and commu- 
nity participation in a setting that involves multiple stake- 
holder groups. 

Expert involvement was further broadened by the inclu- 
sion of nine international CM research experts who did 
not participate in the workshop, but who contributed to 
the survey and both Delphi rounds. The consensus 
process was finalized after feedback from all workshop 
participants and the external review experts. 

Results 

The results of the consensus process are presented in two 
sections: I) Using available data, II) recommendations for 
future clinical studies. 

I) Using available data 

1) Clinical Health Records 

a) Data from clinical health records are 
important and can be very useful for 
generating hypotheses for future randomized 
trials, identifying common diagnostic patterns 
and promising interventions, assisting 
recruitment for prospective studies, correlating 
CM diagnoses with outcomes, and providing 
information on patient characteristics and 
interventions in usual care. 

b) The potential for CER based on clinical health 
records that document CM use (for example, 
Kaiser Permanente California [10,11], Veterans' 
Administration [12]) needs more exploration. 
The value of such records depends on the 
documentation structure and the availability of 
data on useful surrogate outcomes such as 
visits, days off from work, drug use, and/or 
costs. 

2) Data from previous trials 

a) Databases provide helpful information on 
existing CM trials (for example, Oregon 
College of Oriental Medicine for Acupuncture 
(AcuTrials 8 ) [13], New England School of 
Acupuncture (NESA) Database [14], Chinese 
BioMedical Literature Database (CBM) [15], 
China Network Knowledge Infrastructure 
(CNKI) [16], and Chinese Scientific Journals 
Database (VIP) [17]). 

b) There is a need to develop a database of CM 
trials that have included multi-component 
treatments. 

c) Patient level raw data from randomized trials 
on CM should be shared in order to generate a 
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database; this should be especially encouraged 
for all new trials. Guidelines to facilitate such 
sharing should be developed and implemented. 
II) Recommendations for future clinical studies 

The recommendations are summarized under the 

following headings: 

• Designing CER studies 

• Treatments, expertise and setting 

• Outcomes 

• Study design and statistical analyses 

• Economic evaluation 

• Publication 

Design and reporting guidelines relevant to these rec- 
ommendations are summarized in Table 1. A checklist 
highlighting the most important elements is presented in 
Table 2. 

Designing CER studies 

1) Stakeholder engagement 

a) Involvement of all relevant stakeholders (for 
example, practitioners, clinicians, patients, payers, 
researchers) is highly relevant for CER when 
identifying research questions, planning, and 
designing the study, and interpreting study results. 

b) Stakeholder involvement should follow a 
systematic approach (for more information on 
suitable methods see Deverka et. al. [30]). 

2) Study question and the efficacy-effectiveness 
continuum 

a) The study question should be clearly phrased, 
and include all relevant information about study 
participants, interventions, comparison groups 
and outcome parameters. In particular, it should 

Table 1 Relevant guidelines for design and reporting 



clarify whether the CM treatment is to be assessed 
as an "alternative" in direct comparison, using a 
superiority or non-inferiority hypothesis, or as an 
adjunct to a usual or standard care treatment, 
b) During the trial planning phase, time should be 
given to discuss and determine the trial's position 
along the efficacy-effectiveness continuum [31]. 
Use of the PRECIS tool to support this process is 
recommended [32]. 

3) Study designs for complex multi-component CM 
interventions 

a) Pragmatic trials may be used to compare complex 
multi-component treatment alternatives 

(for example, multi-component CM treatment, 
consisting of acupuncture, dietary advice and 
qigong compared to multi-component conventional 
treatment, consisting of prescribed recommendations 
for exercise and diet for the treatment of mild 
hypertension) [20]. 

b) Multi-arm trials may help to identify dosing effects, 
synergistic effects when combining different CM 
interventions, and effective components within one 
treatment modality (for example, isolating 
meditative and breathing components from 
comprehensive qigong/ tai chi protocols). 

c) The complexity of therapeutic decision-making 
and treatment changes within the treatment 
process could be reflected by designs as demonstrated 
by Ritenbaugh et. al. [33,34]. 

Study population 

4) General eligibility criteria 

a) In the context of available resources, eligibility 
criteria should be as broad as possible. The 



Document 


Design 


Reporting 


Reference 


Effectiveness guidance document for acupuncture research 


X 




[9] 


SPIRIT for content of clinical trial protocols 


X 


X 


[18] 


CONSORT for parallel group randomized trials 




X 


[19] 


CONSORT extension for pragmatic trials 




X 


[20] 


CONSORT for non-pharmacological trials 




X 


[21] 


CONSORT extension for cluster randomized trials 




X 


[22] 


CONSORT extension for acupuncture trials 




X 


[23] 


CONSORT extension for herbal interventions 




X 


[24] 


CONSORT extension for non-pharmacological treatment interventions 




X 


[25] 


CONSORT extension for traditional Chinese medicine 




X 


[26] 


CONSORT extension for patient reported outcomes 




X 


[27] 


Guidelines for randomized controlled trials investigating Chinese herbal medicine 




X 


[28] 


Extending the CONSORT statement to moxibustion 




X 


[29] 



CONSORT = Consolidated Standards of Reporting Trials. 
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Table 2 Checklist for the most relevant aspects of 
comparative effectiveness research for Chinese medicine 
clinical studies 

Designing comparative effectiveness research (CER) studie 

1. Stakeholder All relevant stakeholders are involved in 
involvement identification of research topic, plan and design 

of CER, interpretation of results 

2. Efficacy-effectiveness Location on the efficacy-effectiveness continuum 
continuum is determined for participant selection/eligibility 

criteria, treatment protocol, practitioner expertise, 
outcomes, and setting in which the study is 
conducted 



Table 2 Checklist for the most relevant aspects of 
comparative effectiveness research for Chinese medicine 
clinical studies (Continued) 



3. Study design 

Study population 

4. Eligibility criteria 



5. Diagnoses 



6. Patient recruitment 



Designs for multi-component interventions 
should be considered 



Should be as broad as possible in the context 
of available resources - Study population 
includes both CM-naTve and CM-non- naive 
patients 

Recruitment of patients should follow Western 
diagnoses - CM diagnoses should be done 
whenever possible 

Patients are recruited from site(s) where the 
treatment is usually provided 



Treatment, expertise, and setting 

7. Defining treatments 



8. Acupuncture 

9. Qi gong/tai chi 

10. Herbal medicine 

1 1. Treatment 
documentation 

Outcomes 

12. Measures 



13. Timing 



If intervention involves multi-component treatment 
the combination should be plausible and feasible 
in usual care - non-CM best practice alternatives 
are based on guidelines or broad expert consensus 

See [9] 

Style and setting should reflect typical 
community-based programs 

Local and national regulations should be taken 
into account 

Documentation reflects which treatments were 
received by all groups (interventions and 
co-interventions) 



Widely accepted or standardized outcome 
measure used - secondary outcomes capture 
relevant patient-centered dimensions for the 
condition under study 

Assessment schedule is balanced allowing study 
to acquire relevant data without substantial 
disruption of treatment or setting 



Study design and statistical analysis 

14. Allocation 



15. Blinding 



16. Preferences/ 
expectation 

17. Sample size 



Allocation is concealed - stratification for 
subgroups and/or dynamic allocation for key 
characteristics are used 

Outcome data are kept inaccessible to 
practitioners - blinded outcome rater is used if 
possible 

Preferences and expectations are measured at 
baseline 

Sample size takes patient heterogeneity into 
account - required sample size is feasible - Study 
has enough power for planned subgroup 
analyses 



18. Subgroups 



19. Statistical analysis 



Relevant subgroups are pre-planned - exploratory 
subgroup analysis is mentioned in study aims 

Intention-to-treat analyses are planned - relevant 
subgroup analyses are planned - data analyses 
are adjusted for stratification variables, baseline 
differences and relevant confounders 



Economic evaluations 

20. Relevance 

21. Methodological 
approach 



22. Observation time 



Publications 

23. Guidelines 



Setting reflects reality in clinical practice 

Standard methods for economic evaluations are 
used - sensitivity analysis is employed for all 
relevant stakeholder perspectives - relevant 
subgroups are identified 

Long-term observation (>12 months) are 
planned if possible 



Relevant guidelines (CONSORT) are consulted 
and followed 

24. Content Statements of how and why this is CER are 

included - study setting (including practitioner 
selection procedure) is described in detail - 
treatment group description from informed 
consent is provided - comparison groups are 
described in detail - data are provided on all 
interventions and co-interventions received - relevant 
subgroup analyses are reported 

CM, Chinese medicine; CONSORT, Consolidated Standards of Reporting Trials. 



criteria should reflect the evidence of the pattern 
of usage and disease burden, and the study 
population should reflect all well-known relevant 
disease characteristics that may interact with the 
treatment (for example, gender, disease stage, 
comorbidities, co-medications). 

b) Patients with comorbidities should not be 
explicitly excluded from the study enrollment 
unless the comorbidities make them 
inappropriate candidates for the treatment, but 
safety and regulatory aspects have to be taken 
into account (for example, when using herbs). 

c) Both CM naive and non-naive patients should 
generally be considered eligible for study inclusion 
to reflect real-world patient population. If special 
groups are targeted, the rationale should be 
provided. 

5) Diagnosis 

a) The study disease/condition should be defined as 
clearly as possible from the Western medical 
approach as well the CM approach. 

b) In general, recruitment of patients should initially 
follow the Western diagnostic approach. 

c) CM diagnoses should subsequently be made in all 
treatment groups (before randomization in 
randomized studies) and documented whenever 
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possible. A clear, comprehensive and well 
recognized categorization system for the CM 
diagnoses should be used. The CM diagnoses 
could be used for exploratory analyses and to 
generate hypotheses for future studies (for 
example, impact on the outcome, correlation with 
genomics, prediction of the course of disease and 
response to treatment in the different treatment 
groups). 

d) Designing the study around CM diagnoses in 
general should be avoided (for example, including 
only patients with one type of CM diagnosis), 
because results are difficult to implement into a 
medical system that follows a Western approach. 
However, if a clear and limited number of CM 
diagnoses exist, and research-based evidence is 
available to suggest that the CM diagnoses are 
reliable, consistent across practitioners, and have 
a relevant influence on the intervention or the 
outcome, this should be taken into account, 

(for example, by stratified randomization according 
to CM diagnoses, or different treatments for 
different CM diagnoses). 

e) If the CM diagnosis is relevant to the study 
design (for example, preplanned confirmatory 
subgroup analyses according to the CM 
diagnoses), emphasis should be placed on 
strengthening its reliability (for example, assessing 
inter-rater agreement, providing training and/or 
calibration of practitioners). 

6) Patient recruitment 

a) Recruitment should be carried out systematically 
and from various sources (for example, in places 
where the population suffering from the disease is 
available and/or relevant treatments are usually 
employed). 

b) Patients' treatment should be recorded at baseline 
and efforts should be made to recruit both those 
who express treatment preferences and those 
who do not. 

c) Appropriate strategies to ensure successful 
recruitment should be developed and, if possible, 
pretested. 

d) In non-randomized studies, as far as possible, 
recruitment strategies should be similar for all 
treatment arms. 

Treatment, expertise and setting 

7) Defining treatment groups 

a) The treatment alternatives (CM treatments and 
non-CM treatments) should each provide value 
to the patient by having the potential to be "best 
practice" [7]. In the absence of a clear evidence 



base "best practice" of CM can be derived by 1) 
reviewing alternatives that have been effective in 
addressing similar issues in the past and could be 
applied to a current problem, and 2) integrating 
information from a number of sources 
(recommended research protocols, existing 
clinical data, reference to classical usage, and 
formal consensus procedures). If a direct 
"head-to-head" comparison is used, all treatment 
options should reflect usual care as much as 
possible, and ideally the extent of standardization 
should be similar in all treatment groups. If CM 
treatment plus usual care is compared to usual care 
alone, usual care should be similarly defined and 
provided in both groups. 

b) The comparison treatments and their complexity 
should have widespread clinical acceptance 
relative to the condition studied and stage of 
disease. 

c) The rationale for the CM treatment should follow 
one of the following two approaches: 

o If there is positive evidence for single 
treatment components from clinical studies, 
these components may be combined to 
evaluate the comparative effectiveness of a 
complex multi-component treatment. The 
combination of treatment components should 
be plausible and feasible in a routine care 
context and not contradicted by CM theory. 

o If convincing clinical evidence is not available 
for the single treatment components, overall 
effectiveness of a multi-component CM 
treatment should be evaluated before component 
efficacy is addressed. The rationale for the 
treatment components and their combination 
should be based on both thorough clinical 
evidence and associated theory from the 
literature. 

d.) Non-CM comparator treatments should be based 
on evidence, guidelines or broad expert 
consensus. 

8) Special aspects for acupuncture have been described 
in the EGD for acupuncture research [9]. 

9) Special aspects of qigong/tai chi 

a) A large number of Qigong/Tai Chi styles exist 
and the chosen style(s) for a research trial should 
represent to some extent the real world 
heterogeneity of practice in the country where 
the study is performed. 

b) If a very specific style is used, it must be justified 
(specific style or protocol found to be effective in 
prior studies, or potential for widespread 
adoption), and limits to generalizability should be 
acknowledged. 
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c) The setting in which qigong and tai chi is offered 
should be accessible and reflect typical 
community-based programs; the longer-term 
(post-study) sustainability of the program should 
be considered (access to training, classes or 
instruction). 

d) qigong/ tai chi should be provided by qualified 
instructors, with expertise in both protocol 
content and teaching. Training and teaching 
qualifications of all instructors should be 
reported. In studies of high-risk populations, 
treatment safety should play a prominent role, 
using more expert instructors and protocols 
validated with respect to safety. 

e) Studies should provide information that specifies 
exercises (names and style), dose (number and 
duration of classes, home exercise) and ancillary 
training materials offered (for example, books and 
audio-visual material). 

10) Special aspects of herbal medicine 

a) Study designs must comply with local and 
national regulations regarding herbal medicines in 
the country of the clinical trial. Although widely 
used in everyday practice, and in spite of the fact 
that research is urgently needed, research on 
individualized, multi-herb formulations is very 
difficult to accomplish in most Western countries 
due to government regulations and Institutional 
Review Board approval. 

b) The treatment should be based on existing 
evidence (systematic review of Chinese as well as 
European and US -based databases, survey of 
normal practice, practitioner case records, etc.), 
should have "model validity" within CM, and 
provide a rationale for "good practice" in CM. 

c) It must be assured that the treatment does not 
include any endangered species. 

d) Herbal formulations that include different herbs 
need to be adequately defined (chemically to 
assure quality of herbs). 

e) Additional safety aspects have to be taken into 
account including the aspect that relevant 
laboratory tests should be performed during the 
trial to check for potential unwanted side effects 
interactions with other medicines, although the 
latter can impact study design, cost and 
recruitment. 

f ) Relevant aspects for the different stakeholder 
groups should be taken into account when 
planning the treatment. This includes 
consideration of: 

o Different preferences among different 
stakeholder groups for the route of 
administration (for example, decoction, 



granules, or tablets). However, when making 
changes to improve participant compliance, 
care should be taken to maintain treatment 
regimes and dosages that agree with standards 
of "good practice", 
o Costs of the treatment, for the research trial 
per se and for the likelihood of trial results 
informing clinical practice. Cost may be 
affected by the complexity of the herbal 
formula, as well as the dosage and route of 
administration. These are important 
considerations for payers and patients who 
often pay out-of-pocket. 

11) Treatment documentation 

a) All treatments (study treatment, co-treatments, 
over-the-counter self-medication) carried out in 
all groups should be documented as well as the 
context in which the usual care is provided. A 
variety of documentation methods and sources 
may contribute details (including medical records, 
case report forms, patient diaries). 

Outcomes 

12) Measures 

a) The main outcome measures should be 
patient-centered and, if appropriate, include 
relevant biological measures for the respective 
disease/condition. Whenever possible, 
diagnosis-specific validated standards for outcome 
measures published by professional associations 
(for example, the International Headache Society 
on headache measures) [35] or broad published 
expert consensus (for example, on back pain 
measures) [36] should be followed to permit better 
comparison of study results. 

b) If no standard measures for targeted outcomes 
exist, or those that are available are not suitable 
for CER on CM, new outcome measures should 
be developed. Stakeholders should be involved in 
this process and qualitative and quantitative 
research methods should be used in combination. 

c) Multiple primary outcomes addressing distinct 
dimensions may be used, if appropriate. The use 
of multiple primary outcomes should be 
addressed in the sample size calculation and 
statistical analysis plan. 

d) Secondary outcome measures should capture 
relevant patient-centered dimensions of the 
respective disease/condition (both self-reported 
and biological, as well as outcomes relevant within 
the CM model for the treated CM diagnoses). 
Additional secondary outcomes might, if appropriate, 
include measures of collateral effects (i.e., positive 
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and negative consequences of the treatment 
experience, often seemingly unrelated to the main 
outcomes). 

13) Timing 

a) Outcomes should be evaluated over a sufficient 
long period to capture true impact on chronicity 
of disease and to distinguish this from short term 
or intermittent relief. 

b) The use of multiple intervals to document and 
compare the trajectory and persistence of 
treatment effects is recommended. Data 
collection methods that do not have a direct 
influence on the treatment plan (for example, text 
messages, phone calls, smart phone applications) 
are recommended. However, the frequency of 
assessment should be balanced, so that relevant 
information is gained without major disruptions 
of treatment implementations or practice setting 
and with minimal risk of respondent overload. 

Study design and statistical analysis 

14) Allocation methods 

a) Use of appropriate allocation methods is strongly 
recommended. Randomization at the level of 
individual patients is still the most frequently 
used method, but dynamic allocation procedures 
(for example, rank minimization) may be used as 
an alternative. The final choice depends on the 
design of the study and the sites at which the 
study will be conducted [37]. 

b) Stratified randomization or adaptive allocation 
techniques may be used to prevent imbalances 
for relevant covariates and potential confounders 
in study arms [38,39]. 

c) Partially randomized patient preference designs 
have an advantage in that they provide additional 
exploratory information as to whether the results 
observed for randomized patients are different 
from those who were not randomized because of 
treatment preferences. However, these designs, 
while adding potentially important outcome data 
to a clinical trial, are often not feasible because of 
the need for much larger sample sizes and higher 
costs [40]. 

d) Cluster randomization is the best approach under 
circumstances where the randomization of social 
units (for example, clinics) is advisable to avoid 
contamination of treatments between groups. 
When planning such a trial, it is necessary to 
consult the relevant literature and local 
institutional roles to determine from whom, 
when, and how informed consent must be 
obtained [41], and to take into account that a 



larger sample might be needed than in patient 
level randomized trials [42], because the trials are 
powered based on the number of participating 
units. 

e) Standard procedures ensuring allocation 

concealment (for example, central randomization 
or secure databases) should be employed. 

15) Blinding 

a) Blinded outcome measurement (for example, a 
blinded rater) is recommended in order to reduce 
bias, especially for outcomes that, in usual clinical 
practice, are assessed by the practitioner (for 
example, physical assessments). Methods to 
minimize the risk of unblinding (for example, 
allocation concealment, rater training, 
standardized assessment protocol) should be 
employed. 

b) Data analyses should be blinded whenever 
possible. 

c) Outcomes data reported by patients for the study 
purpose (for example, quality of life assessment) 
should be kept inaccessible to the practitioner 
(for example, by using sealed envelopes or 
preferably by sending questionnaires directly to a 
study office independent of the study site or using 
a blinded interviewer). 

d) Recommendations for blinding the treatment (for 
example, when using a double dummy placebo 
for the comparison of herbal medicine with 
conventional drugs) are provided in the 
guidelines developed in the European Union 
funded GP-TCM project [43]. 

16) Patient preferences and expectations 

a) Patient preferences should, if appropriate, be 
acknowledged in the study design, e.g., by using a 
partially randomized patient preference design. If 
such a design is not feasible, then it is important 
to document both the patients' preferences 
regarding the treatment options available in the 
trial as well as the degree of their knowledge and 
experience with these treatment options. 

b) Assessing patient and practitioner preferences 
and expectations for the treatments offered in the 
study at baseline should be considered. In 
randomized trials they should be assessed before 
randomization and for all available treatment 
options. 

17) Sample size 

a) Sample size should focus on the main outcome(s) 
and the minimum clinically important difference 
(MCID) for the respective outcome(s) and take 
into account greater heterogeneity in CER study 
populations. Because of this, researchers should 
specifically avoid conducting small trials (< 50 
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patients per arm) in CER, unless there is a 
specific reason to do such studies (for example, 
pilot studies to test feasibility and recruitment). 

18) Subgroups 

a) Relevant subgroups for the disease/condition 
under study should be identified based on 
existing data and the literature. Also of current 
interest are subgroup analyses for different CM 
pattern diagnoses and for CM patients who are 
naive/non-naive to CM. If sample size permits, 
further analyses can be carried out for gender, 
age, ethnicity, disease severity/duration, treatment 
preference and recruitment site. 

b) The main subgroup analyses should be 
pre-specified in the analysis plan and included in 
sample size planning for confirmatory testing. 
Further subgroup analyses can be done on an 
exploratory level, but should be stated as an 
objective in the study protocol. 

19) Statistical analysis 

a) Primary analysis for superiority trials of CM 
should be pre-specified and intention-to-treat. 
In order to assess real-world effectiveness of 
treatments, benefits and harms should be compared 
in relation to the treatment to which patients were 
assigned. 

b) Analyses should adjust for relevant potential 
confounders (for example, baseline value of the 
outcome measure, stratification variables, 
expectation, and baseline CM diagnosis). 

c) Especially in non-randomized studies, procedures 
to compensate for baseline differences must be 
used (for example, matching and/or adjusted 
analysis). 

Economic evaluations 

20) Relevance 

a) Comparing the effectiveness of treatment options 
should be the primary aim of CER, but economic 
evaluations should be included whenever possible 
as a secondary aim. 

b) To allow realistic cost estimates, the setting(s) of 
the study should reflect the real-world clinical 
practice for each treatment as closely as possible. 
If a study includes a standardized and a 
non-standardized CM arm, it would be useful to 
compare their cost-effectiveness. 

21) Methodological approach 

a) Standard effectiveness measures for economic 
evaluations should be employed that include both 
benefits and harms (for example, utility measures 
based on SF-36, SF-12 or EQ-5D) [44]. 



b) Economic evaluations should be designed to 
reflect stakeholder perspectives with sensitivity 
analysis performed, whenever possible, from 
different stakeholder perspectives (for example, 
society, payer and patient). Because CM is often 
paid out-of-pocket, the patient's perspective is 
highly relevant. 

c) Requirements of the local context (for example, 
guidelines by regulatory agencies) should be taken 
into account. 

d) Subgroup analyses should mainly focus on 
subgroups defined a priori for the effectiveness 
study. Additional analyses should be clearly 
described as exploratory. A subgroup analysis for 
gender is recommended since there is preliminary 
evidence that gender may influence the 
cost-effectiveness of CM treatment [45]. 

e) Exploratory analyses of factors that predict a 
better cost-effectiveness are suggested to develop 
future hypotheses. 

22) Observation time 

a) Long-term observations with intermediate 
measurement time points are highly 
recommended for economic evaluations of 
chronic disease in order to evaluate development 
of cost-effectiveness over time. 

Publication 

23) Existing guidelines 

To ensure that CER on CM will fulfill reporting 
standards, the relevant CONSORT guidelines should 
be followed (see Table 1). 

24) Content 

a) Publication of a detailed study protocol (design 
publication) should take place whenever possible 
prior to the recruitment of the last patient. 

b) The study should be registered in an 
internationally accessible trial database with as 
many details as possible provided. 

c) Publication of the completed study should 
describe why and how it qualifies as CER and 
make clear the phase of the study. 

d) The setting of the study should be described, 
including information about the typical care 
setting in the country where the study was 
performed (and, if relevant, in other countries). 
The procedure for selection of practitioners for 
each treatment group should be described, with 
an account of whether and how those included in 
the study differ from the average practitioner (for 
example, training, experience). 

e) Information on how patients were informed 
about the treatment options should be provided. 
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f) If a usual-care or standard-care comparison 
group is used, a detailed description with citations 
for standard care should be included in the 
intervention section. 

g) Detailed results of all treatments should be 
presented; adherence to interventions and 
co-interventions should be reported for each group. 

h) Whenever possible, the most relevant subgroup 
analyses and analyses of patient characteristics 
that predict a better outcome should be published 
together with primary results. Detailed subgroup 
analysis and/or de-identified patient level data can 
be provided as online files. 

Discussion 

This is the first EGD for clinical research involving a com- 
plex and multi-component medical system, providing de- 
tailed advice for the design of CER in the field of Chinese 
medicine for single as well as multi-component treat- 
ments. This EGD has been derived from a systematic de- 
velopment process, with active involvement of different 
stakeholders from the West and China, and aims to in- 
form researchers inside and outside China when designing 
their trials. The involvement of China-based stakeholders 
reflects both the geographic roots of CM and a growing 
interest in CER studies in China. During the development 
process, stakeholder groups uncovered a broader under- 
standing of the complexity of a multi-component treat- 
ment, the cultural differences in CM practice and research 
between China and other countries, and the resulting 
challenges for the study design. The heterogeneity of CM 
as practiced in different countries made it necessary to de- 
velop recommendations that account for these variances 
of style and context. China has a strong research focus on 
herbal medicine, however this is less common in other 
countries due to regulatory requirements. Herbal medi- 
cine trials have unique challenges and within this consen- 
sus process we were only able to discuss the most 
prominent ones. For CM herbal medicine trials, it is rec- 
ommended that the guidelines for randomized controlled 
trials investigating Chinese herbal medicine be utilized 
[28]. A limitation of consensus procedures is that not all 
aspects of the study design can be addressed. For example, 
no recommendations for the study sites were discussed. 

There was discussion about the adequacy of compari- 
son groups, but because of the broad range of optional 
research questions and the multifold combination of in- 
terventions, no detailed recommendations were made. 
However, there was consensus that the comparison 
group(s) should have the option for best practice and 
should be based on guidelines or broad expert consen- 
sus as recommended under point 7. 

Within the process, several methodological aspects 
unique to CM were identified that need further research 



and clarification. For example, the CM diagnosis classifica- 
tion system and the heterogeneity of its application need 
research to ensure overall validity and reliability. Another 
example is that the CM treatment benefits should also be 
measurable in CM terms. This goal may be complicated 
by aspects of CM's explanatory model that aims to restor- 
ing balance or increasing resilience, for which suitable out- 
come measures are not yet developed. 

Conclusion 

Although CONSORT statements provide guidelines for 
reporting studies, EGDs provide recommendations for the 
design of future studies and can contribute to a more stra- 
tegic use of limited research resources as well as greater 
consistency in trial design. In particular, the present EGD 
provides the first set of systematic methodological guid- 
ance for future CER on CM. 
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